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Intelligent Instructional Systems for Teaching Procedural Skills 

PUBLICATIONS 

Feurzeig, W., and Ritter, F. (1987) "Understanding Reflective Problem Solving". In Intelligent 
Tutorin K Sistems: Lessons Learned, ed. Psotka. J., Massey, D., Mutter, S., Lawrence Erlbaum 
Associates, Hillsdale, N. J. 

Abstract 

ICAI systems typically acquire knowledge about a student's misconceptions and procedural bugs by 
making inferences based on the student's problem-solving actions. Thus, in work oh electronic circuit 
troubleshooting problems, the student's knowledge and difficulties are inferred from observations of the 
tests and measurements he makes along the way. In this paper we propose the alternate strategy of 
acquiring information directly from the student. This strategy has been implemented as ah instructional 
monitor within the QUEST system for training electrical system troubleshooting. Every time the student 
calls on the QUEST simulator to carry out a test or make a circuit measurement, the monitor asks the 
student the reason for his action. After the simulator performs the requested acuon. the monitor asks the 
student what he learned from the test or measurement. Student responses are made by selecting items in 
response windows and pointing at circuit components or subcircuits. The approach can yield extensive 
information about the student's expectations, theories, and plans as a foundation for making inferences 
about his misconceptions and bugs. This strategy is therefore proposed as a valuable complement to pure 
inferencing methods. The detailed operation of the monitor is described and the transcript of a QUEST 
troubleshooting interaction under this mode of operation is shown. 



Frederiksen. J R. & White. B.Y. (1987) "Intelligent Tutors as Intelligent Testers". In Diagnostic 
Monitoring of Skill and Knowledge Acquisition. Frederiksen. N.. Glaser. R.. Lesgold. A.. & Shafto. 
M. (Editors). Lawrence Erlbaum Associates, Hillsdale, New Jersey. 

Abstract 



This paper explores the use of intelligent tutoring systems in the assessment of students' knowledge 
bases, problem solving processes, and learning strategies. We describe a fonn of tutoring system that is 
based upon a progression of cognitive models that reason and solve problems within a domain. Such 
model progressions can provide runnable simulations o' students' mental models, which can be used to 
determine not only what problems a student is capable of solving at a given point in training, but also 
what additional concepts and skills a student must acquire in order to solve problems that he or she 
currently cannot solve. Since tutoring systems of this class can support different learning strategies, it 
also becomes possible to determine the student's preferred mode of learning, or to assess his or her'rate of 
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learning when a particular learning strategy is imposed on the student. Using such measures derived 
within the tutoring environment: it should be possible to predict learning outcomes hot drily within the 
tutoring system, but also in other learning environments and oh the job. 



White, B.Y. & Frederiksen, J.R. (1987) "Qualitative Models and Intelligent Learning 
Environments". In AI and Education . Lawler, R. & Yazdani; M. (Editors). Ablex t>ublishing 
Corporation, Norwood, New Jersey. 

Abstract 

One promising educational application of computers derives from their ability to dynamically 
simulate physical phenomena. Such microworlds permit students to explore; for instance; electrical 
circuit behavior or particle dynamics. In the past, these simulations have been based upon quantitative 
inodels. However, recent work in artificial intelligence has created techniques for basing such 
microworlds on qualitative reasoning. Qualitative models not only simulate the phenomena of the 
domain, but also can generate explanations for the behavior under study. Sequences of such models, that 
attempt to capture a progression from novice to expert reasoning, permit microworlds to incorporate 
features of intelligent tutoring systems. The learning environment can embody the model progression by 
making available to students (1) microworlds of increasing complexity, (2) problems classified by the 
level of model required for their correct solution; and (3) explanations focused on the differences between 
models and their predecessors. Students can then utilize these microworlds, problem sets, and 
explanations to aid them in developing ah understanding of the domain. 



Feurzeig, W. (1986) "Developing an Exportable ICAI Technology". In the Proceedings of the 
Workshop on Introducing Intelligent Computer Assisted Instruction into Military Training, U.C.L.A., Los 
Angeles, CA., U. &. Army Research Institute Technical Report, Arlington. VA. 

Abstract 

The instructional capabilities of current ICAI systems and the inherent difficulties in making 
intelligent inferences about a student's knowledge and underlying misconceptions, as illustrated by the 
student modeling work in systems such as WEST and DEBUGGY, are summarized. The teaching of 
qualitative thinking and the use of qualitative simulation models for producing understandable 
explanations, in systems such as Steamer and QUEST, which support student exploration of system 
behavior and student practice in system operation and troubleshooting, is described. The paper calls for 
the development of versatile training systems that integrate facilities for sti<dent-directed exploration and 
practice with tutorial ICAI programs. These systems would be designed to support several modes of 
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instruction including articulate experts with capabilities for demonstrating and explaining task 
performance, critics for giving students very specific diagnostic feedback following practice, and 
intelligent microworlds for guiding student exploration and inquiry. 



Ritter, F. (1986) "OREO - Adding Orientation to a Dynamic Qualitative Simulation". BBN 
Technical Report, October. Submitted to Third International Conference on Artificial Intelligence and 
Education, University of Pittsburgh. 

Abstract 

In order to simulate electrical circuits accurately using a qualitative model, it is necessary to give 
parts an orientation with respect to the voltage source in the circuit. Mistakes made by beginning students 
of electricity, such as not finding a short across a part, can often be attributed to incompletely orienting 
die circuit. Experienced engineers orient circuits successfully, though they Usually do not indicate 
orientations in an explicit way on their circuit diagrams. This paper presents an algorithm for orienting 
circuits Sat is based on the way engineers orient circuits. The algorithm can explain its operation. It can 
recognize shorts, bridge elements, and other paths useful for a quantitative simulation. The algorithm 
allows qualitative systems to be dynamically altered and is upwardly compatible with quantitative 
simulations. 



White, B.Y. & Frederiksen, J.R. (1986) 'Intelligent Tutoring Systems Based Upon Qualitative 
Model Evolutions". In the Proceedin gs of the Fifth National Conference on Artificial Intelligence , 
Philadelphia, Pennsylvania. 



This paper is incorporated within the book chapter referenced above. It provides an overview of the 
rationale underlying the design of intelligent tutoring systems based upon model evolutions. It also goes 
into some depth to describe the qualitative causal models that we created for instructional purposes. 




White, B.Y. & Frederiksen, J.R. (1986) "Progressions of Qualitative Models as a Foundation for 
Intelligent Learning Environments". Report No. 6277, BBN Laboratories, Cambridge, Massachusetts. 

Abstract 

The design of our intelligent learning environment is based upon a theory of expertise and its 
acquisition. We find that when reasoning about physical systems, experts utilize a set of mental models. 

8 
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For instance, they may use qualitative as well as quantitative models, and behavioral as well as functional 
models. Hie transition from novice to expert status can be regarded as a process of model evolution: 
students formulate a series of upwardly compatible models, each of which is adequate Tor solving some 
subset of problems within the domain. Further, students need to evolve hot just a single model, but rather 
a set of models that embody alternative conceptualizations of the domain. Finally, we claim that in the 
initial stages of learning, students should focus on the acquisition of qualitative models: quantitative 
models should be introduced only after the domain is understood in qualitative terms: 

In this article, we focus primarily on qualitative, behavioral models of electrical circuit operation 
designed to make the casuality of circuit behavior derive clearly from basic physical principles, the 
constraints on model evolution, in terms of causal consistency and learnability, are discussed and a 
sequence of models that embody a possible transformation from novice to expert status is outlined. 

The learning environment we have constructed lets students solve problems, hear explanations, and 
perform experiments, all in the context of interacting with a dynamic simulation of circuit behavior. 
However, unlike most simulations, the underlying model is qualitative not quantitative. Further, the 
simulation is performed not by a single model, but rather by a progression of models that increase in 
sophistication in concordance with the evolution of the students' understanding of the domain. 

_ _ Viewing instruction as producing in the student a progression of models permits a tutoring system 
architecture with elegant properties. Within our system, the student model, the tutor, and the domain 
simulation are incorporated within the single model that is active at any point in learning. This model is 
used to simulate the domain phenomena, is capable of generating explanations by articulating its 
behavior, and furnishes a desired model of the students' reasoning at that particular stage in learning, the 
progression of models also enables the system to select problems and generate explanations that are 
appropriate for the student at any point in the instructional sequence, tn order to motivate students to 
transform their models into new models, they are given problems that the new model can handle but their 
present model cannot. This evolution of models also enables the system tc focus its explanations on the 
difference between the present model and the hew model. 

Such a system architecture also permits a variety of pedagogical strategies to be explored within a 
single instructional system. Since the system can mm a problem into an example by solving it for the 
student, the students' learning can be motivated by problems or by examples. That is, students can be 
presented with problems and only see examples if they run into difficulty; alternatively, they can see 
examples first and then be given problems to solve. Also, by working within the simulation environment, 
students can use a circuit editor to construct their own problems and thus explore the domain in a more 
open ended fashion. The system is capable of generating runnable qualitative models for any circuit that 
the student or instructional designer might create. Further, the learning process can be managed either by 
the system or by the student. For example, students can be given a map of the problem space and can 
decide for themselves what class of problems to pursue next or even what pedagogical strategy they want 
to employ. 
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Feurzeig, W. (1985). "AI Aids for Design and Automation of CAI Programs". In the Proceedings 
of the Workshop on Knowledge Acquisition for Computer Assisted instruction, Carnegie Mellon 
University, U.S. Army Research Institute, Arlington, VA. 

Abstract 

The problem of preparing high quality computer assisted instruction in areas requiring knowledge 
intensive skills is discussed. An Al-based approach to aid instructional designers in automating the 
generation of such instruction is presented. Specific tools and facilities for implementation of several 
modes of instruction, and for testing and editing of the prototype instructional materials, are described. 
These methods are designed to be specifically applicable to instruction in the operation and maintenance 
of complex engineered devices, in tasks such as electronic troubleshooting and hydraulic power plant 
operation. 



Feurzeig, W: (1985) "Cognitive Science, Artificial Intelligence and Complex Training". In the 
Proceedings of the NATO Symposium on Computer-Based Instruction in Military Environments, 
Brussels, Belgium, ed: R. Seidel & P. Weddle, Plenum Press, approx. March,1987. 

Abstract 

Ah effective program of instruction directed toward the acquisition of complex cognitive skills 
heeds to balance prescriptive tutorial modes with instructional activities that support exploration and 
practice. This presentation argued that current research on intelligent instructional systems has a strong 
prescriptive bias, a one-sided focus emphasizing the development of directive tutorial methods and often 
ignoring those complementary components of a complete learning system which allow the student greater 
initiative and control. Current work on modelling student knowledge in intelligent tutoring systems was 
reviewed, lie difficulties in diagnosing students' misconceptions, conceptual gaps, and procedural bugs 
were described: The fundamental instructioanl problem of how the knowledge of a student's 
misconceptions and bugs can be effectively used to aid the student overcome them and learn the skill 
being taught, was discussed. 




Frederiksen, JJR. & White, B.Y. (1985) "A System for Teaching a Qualitative Understanding of 
Electrical Circuit Behavior and Troubleshooting"; In the Proceedings of the Second International 
Conference §S Artificial Intelligence and Education, Exeter, United Kingdom. 
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Abstract 

The goal of this project was to develop an intelligent tutoring system for teaching qualitative 
reasoning about electrical circuits in general, and troubleshooting of automotive electrical systems in 
particular. The system is designed to provide a simulation environment in which students can solve 
circuit problems. It also serves a tutorial function and can be called upon to solve problems and 
demonstrate to students the reasoning involved. The instructional objective is for students to acquire, 
through their exposure to a sequence of carefully selected circuit problems, a sequence of increasingly 
sophisticated models of circuit behavior. 

To this end, we have formulated a series of cognitive models that correspond to different levels of 
understanding of electrical circuit concepts. Each model is upwardly compatible with higher level 
models, but is restricted in the set of circuit problems for which it is adequate. While these mental models 
vary in the complexity of the circuits about which they can reason, they all reoresent "runnable" 
qualitative simulations and enable the student (or the system) to predict the behavior of circuits of a given 
level of complexity. 

Circuit problems given to students include (1) making predictions about circuit behavior, and (2) 
troubleshooting or isolating faults within circuits. Corresponding to each of these two types of problems 
are two tutoring systems: (1) a qualitative, causal simulation of electrical circuits that illustrates principles 
for reasoning about circuits; and (2) an "expert" troubleshooter that demonstrates a strategy for isolating 
faults within circuits and that incorporates the same type of reasoning as that involved in predicting circuit 
behavior. 

These tutoring systems are based upon a qualitative approach to troubleshooting taken by an expert 
mechanic whom we have studied. His methods are based upon the fundamental idea of a circuit: For a 
device to operate, there must be an electrical potential across the device. When such an electrical 
potential exists, a current will flow through the device (provided it is conductive), causing it in some 
cases to change its state. Knowledge of the device, in the form of a device model, allows one to infer 
what state the device will enter and under what conditions. 

When troubleshooting, the expert's goal is to choose a test point that divides the circuit into two 
parts and then to infer which portion of the circuit, that in parallel or that in series with a test light, is 
faulty. When the fault has been isolated to within a portion of the circuit, the expert then moves to a new 
test point within the faulty part of the circuit and recursively applies the troubleshooting logic. The 
approach of our expert has the advantage of being generally applicable to a large class of series-parallel 
circuits. However, it presupposes a knowledge of electrical circuits and requires an ability to reason 
qualitatively about circuits. 

Given the need to teach electrical principles and their implications as a prerequisite to teaching 
troubleshooting, we created an instructional environment that is capable of demonstrating and providing 
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practice in applying circuit principles, the basis of this system is a progression of increasingly 
sophisticated, qualitative causal models that utilize knowledge of the structure of the circuit, the 
functioning ; of the devices within the circuit, and basic electrical principles to simulate circuit behavior; 
These simulations needed to be robust in permitting faults to be introduced or circuits to be modified 
without requiring a new model for each perturbation in the circuit. By utilizing context Free functional 
models for devices, along with topological search processes for evaluating electrical potential across 
devices, we were able to construct circuit models that accurately simulate the behavior of a large class of 
circuits in both faulted and unfaulted states. 

For any given circuit in the teaching sequence, the student is asked to predict the effects of faults 
that the system has introduced, and to locate unknown faults. At any time, the student can call upon a 
model of circuit behavior to explain the functioning of the circuit, or upon a troubleshooting model to 
describe how it would isolate a particular fault in the circuit. Each tutorial program utilizes a model that 
articulates reasoning at a level of explanation that is appropriate for the particular stage in instruction. We 
are currently conducting research to determine whether the set of activities that we have developed, 
combined with the explanatory power of the progression of circuit and trouble shooting models, enables 
students to acquire a qualitative understanding of circuit behavior and of troubleshooting logic. 



White, B Y. & Frederiksen, J.R. (1985) "QUEST: Qualitative Understanding of Electrical System 
Troubleshooting" . ACM SIGART Newslet ter , 93, 34-37. 

Abstract 

In the context of designing a computer-based system for teaching electrical troubleshooting, we 
have developed a qualitative model of circuit behavior and a model of expert troubleshooting. The 
purpose of these models is to explain to students the operation of circuits in faulted and unfaulted 
conditions and to demonstrate troubleshooting techniques. This paper outlines the troubleshooting model 
and the qualitative causal models of circuit behavior from the perspective of the instructional constraints 
that contributed to their design. 



Feurzeig, W., Frederiksen, J., White, B., and Horwitz, P. (1984) "Designing an Expert System for 
training Automotive Electrical Troubleshooting". In the Proceedings of the Joint Services Workshop: 
Ar ti fi cial Intelligence in Maintenance , Air Force Human Resources Laboratory, Brooks Air Force Base, 
Texas, AFHRL-TR-84-25. 

Abstract 

this paper discusses the preliminary version of QUEST, an expert instructional system for teaching 



ERLC 



BBN Report No. 6460 



BBS Laboratf 1 ^ 

Incorporated 



the basic principles of electrical circuit operati 0n and the skills of circuit trdubi«*9ouhg. the 
troubleshooting model which served as the basis for ^ expert, the Fe ed . Device _ Gr0Un d ^ G ) strategy, 
is described together with the other major component of the initial d esign> fte pr0g raJT> V displaying 
Circuit behavior. The system used color colmg to re p r esent voltage, and siraulated m0 U°t\ a j ong wires 
connecting circuit components to repfesent current, „, a dynamic gra phie pres entation d eS kmd to make 
these notions transparent to beginning students. 



White, B.Y. & Frederiksen. J.R. (1984) "Modeling Expertise i n Troubleshooting % Reasoning 
about Simple Electric Circuits". In rr»e Frcveedir^ of die Annua} Mating ^ m c^jtive Science 
Society , Boulder, Colorado. 



act 



In this paper we discuss the psychological criteria for constructing models of troub ,e $hootirig and 
reasoning about circuits. The focus is on determining how models of circuit beha vi^ influence the 
learning of troubleshooting and how training in troubleshooting influences learning «° r $asoh about 
circuits. 
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